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(54) Abstract Title: Transmitting image data processed in accordance with image processing parameters 
received from the receiving device 

(57) Embodiments of the invention are concerned with transmitting image data for review at remote 
locations, in particular with identifying a means of processing medical image data received from a 
plurality of imaging devices. 

Accordingly embodiments provide a method of transmitting medical image data from a first system 
attached to a communications network to a second system requesting medical image data, the second 
system being remotely connected to the first system via the communications network, the method 
comprising the steps of: 

receiving first image data from a medical imaging device, the image data having a video format; 

receiving data identifying one or more image processing parameters, from the second system; 

in response to receipt of said image processing parameters, selectively processing the first image 
data in accordance therewith so as to create second image data at the first system; and 

transmitting the second image data to the second system. 
Embodiments of the invention therefore operate on data having video format which means that they 
advantageously operate independently of codec type, and can be used in conjunction with any type of 
imaging device that is arranged to generate a video signal. 

An independent claim is included for receiving a plurality of data streams and associated image 
processing parameters wherein a display mode is selected from a plurality of modes in dependence on 
the image processing parameters. 
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Method and App aratus for Transmitting Image Data 

Field of the Invention 

The present invention relates to a method and apparatus for transmitting 
image data and, in one aspect, is concerned with remotely transmitting image 
data received from a plurality of disparate imaging devices, and in another 
aspect, is concerned with remotely transmitting image data over bandwidth- 
limited communications channels. Embodiments of the invention are 
particularly, but not exclusively, concerned with processing and transmitting 
medical image data. 

Background of the Invention 

Imaging techniques such as computed tomography imaging (CTI), 
magnetic resonance imaging (MRI), echocardiograph and the like are essential 
tools for diagnosing a range of medical conditions including heart disease, heart 
attacks, sports injuries, mammography and cancer. Current trends in imaging 
include higher resolution, 3D imaging, and faster image capture time, with 
physicians routinely viewing high resolution displays of beating hearts to help 
diagnose cardiovascular disease. 

Such images are typically stored in digital format for viewing on 
computers and the like by means of proprietary software. Transferring such 
files from one file system to another, remote, file system is extremely attractive 
from the point of view of providing an efficient and prompt healthcare service, 
but achieving this within a practically useful time frame is a technically 
challenging problem, due to constraints on available bandwidth. Essentially 
both the quality of imaging, and thus sizes of the files for transmission, are 
increasing, as are demands from other data transfer services, which means that 

fhrrf if m rvar precont nood tn improve, methods of dula transmission: 

International patent application PCT/US98/19065, publication number 
W099/494I2, presents one approach to this problem, namely selecting a region 
or regions of interest in digital image data, and attributing a priority to each 



region. A wavelet transform of the pixel values of an entire image is performed 
in order to obtain a complete set of transform coefficients of the wavelet, 
whereupon those and the transform coefficients corresponding to each region of 
interest are identified. The transform coefficients for these regions of interest 

5 are emphasized either by scaling them up such that more bits are allocated to 
these transform coefficients or by advancing the encoding ordering of these 
coefficients. This technique effectively allows for de-prioritisation of areas via 
the encoding step, which results in a reduction in the overall data rate since less 
information about these de-prioritised areas will be sent. However, a drawback 

) with this technique is that it is only applicable to codecs using wavelet transform 
encoding. 

US patent US 6,314,452 describes another approach, namely a 
progressive image encoding method, where an image is compiled as a scries of 
layers comprising a poor quality base layer and a plurality of other layers, each 
being more and more detailed with respect to one another. Transmission of the 
encoded data involves several stages: firstly transmitting one, or an inilial 
number of, layers, and thereafter selectively transmitting areas of the upper 
higher detail layers in response to a user request. Whilst this reduces bandwidth 
usage, there are nevertheless latency problems associated with receiving 
requests for layer data and sending data in response thereto. 

Summary of the Invention 

In accordance with a first aspect of the present invention, there is 
provided a method of transmitting medical image data from a first system 
attached to a communications network to a second system requesting medical 
image data, the second system being remotely connected to the first system via 
the communications network, the method comprising the steps of: 

receiving first image data from a medical imaging device, the first image 
data having a video format; 

receiving data identifying one or more image processing parameters, 
from the second system; 



in response to receipt of said image processing parameters, selectively 
processing the received image data in accordance therewith so as to create 
second image data at the first system; and 

transmitting the second image data to the second system. 
Embodiments of the invention therefore operate on data having a video 
format which means that they advantageously operate independently of codec 
type, and, unlike the method described in W099/49412 (which works at the 
codec level) can be used in conjunction with any type of imaging device that is 
arranged to generate a video signal. 

Preferably the first image data comprise analogue video format data, and 
the method includes converting the analogue video format data into initial 
digitised data, and selectively processing the initial digitised data so as to create 
said second image data. 

The first image data can comprise one or a plurality of images. 
Advantageously embodiments of the invention can be used to process a plurality 
of still images, such as are generated by an MRI imaging device, and a sequence 
of moving images, such as are generated by an ultrasound scanning device. In 
the case of still images, individual ones of the plurality vary with respect to one 
another spatially, whereas in the case of moving images, individual images vary 
with respect to one another temporally. Thus, for cases where first image data 
. comprise a plurahty of images, embodiments of the invention could be used to 
process individual images that vary spatially and/or temporally with respect to 
one another. 

In at least one arrangement the image processing parameters include one 
or some of frame rate, resolution, and/or data rate, so that processing of the 
received image data effectively involves modifying the bandwidth requirements 
of the second image data. Since the bandwidth requirements can be set on a per- 
image basis this therefore provides an extremely flexible way of controlling 
bandwidth rrquirementc in dopondanco on the nature of the fu&l image UaU mid 
the type of communications network through which the second image data is to 
be transmitted. 



The image processing parameters can additionally include data 
specifying extents of an area of interest within said first image data, which 
enables users of the second system to specify and select different regions within 
the image, and, since frame rate, resolution and/or data rate (generally referred 
to herein as image reproduction parameters) can be specified on a per image 
basis, the speed and resolution of these different images can easily be modified. 

Preferably the second image data comprise a plurality of digitised 
images, and image processing parameters can be specified for each of the 
plurality. This means that users of the second system can view a plurality of 
images, each having different sizes and/or being reproduced at different rates 
and/or resolutions. 

In accordance with a second aspect of the present invention, there is 
provided a first system and a second system arranged to perform the method 
described above. The second system comprises one or more terminals that are 
logically remote from the first system. In at least one arrangement at least one 
of the terminals of the second system can be physically located in close 
proximity to the first system, so that an operator of the first system can review 
the data transmitted to the second system. 

In accordance with a further aspect of the invention, there is provided a 
data processing system for use in displaying medical images, the data processing 
system comprising: 

data receiving means arranged to receive a plurality of data streams and 
image processing parameters corresponding thereto, each data stream 
corresponding to a medical image; 

display means arranged to display a display mode selected from a 
plurality of display modes, the display modes comprising a display mode 
having at least two regions each corresponding to one of said received data 
streams, 

wherein the system is arranged to select a display mode in dependence 
on the image processing parameters. 

In one arrangement the data streams are video data streams and the 
image processing parameters include position, extents (or size of image), frame 



rate, resolution and data rate, so that selection of display mode is dependent on, 
for example, the positions and frame rates of the medical images contained 
within the data streams. 

This aspect of the invention can be seen as relating to processing 
performed at the second processing system, which preferably includes at least 
one multimodal input device. During display of a selected display mode, the 
display means is advantageously arranged to receive data indicative of a portion 
of interest via the multimodal input device, e.g. via a mouse input from a 
physician viewing the medical images. The data processing system can then 
transmit said data indicative of a portion of interest to a terminal located remote 
from the data processing system, which in one arrangement is the first system. 
The first system can then use these image processing parameters to create 

further medical images, and transmit these remotely to the second system. 

Whilst embodiments of the invention are particularly concerned with 

processing and transmission of medical image data, the method could be applied 

to other types of image data. 

Further features and advantages of the invention will become apparent 

from the following description of preferred embodiments of the invention, given 

by way of example only, which is made with reference to the accompanying 

drawings. 

. Brief Description of the Drawin gs 

Figure 1 is a schematic diagram showing an environment in which 
embodiments of the invention operate comprising a medical imaging device, a 
first system and a second system; 

Figure 2 is a schematic diagram showing components of a terminal 
forming part of the first system shown in Figure 1; 

Figure 3 is a schematic block diagram showing, in greater detail, some of 
fhr rnmponentc shown in Figure 2 and the Xluw uf data llieiebetWccii; 



Figure 4 is a flow diagram showing steps performed by the components 
shown in Figure 2 in processing and encoding data captured by the medical 
imaging device shown in Figure 1 ; 



Figure 5 is a schematic diagram illustrating examples of constituent parts 
of image processing parameters; 

Figure 6 is a schematic diagram showing, in greater detail, some of the 
components shown in Figure 2 and the flow of data therebetween; 

Figure 7 is a schematic diagram showing components of a terminal 
forming part of the second system shown in Figure 1; 

Figure 8 is a flow diagram showing steps performed by the components 
shown in Figure 7; and 

Figure 9 is a schematic diagram showing the effects of the steps shown 
in Figure 8 on received video streams. 

Detailed Description of the Invention 

Embodiments of the invention are concerned with controlling the 
processing of image data, which have been captured at an imaging device, by 
terminals located remote from the imaging device. In particular, embodiments 
of the invention are concerned with remotely controlling the format, 
reproduction and content of data that are transmitted to one or more such 
terminals through encoding or re-encoding of data. 

An environment 10 within which the embodiments operate is shown in 
Figure 1, comprising a first system 1 attached to a communications network 5 
and arranged to receive medical image data in video format from a medical 
imaging device 3. The environment 10 also includes a second system 7, which 
is remotely connected to the first system 1 via the communications network 5 
and is arranged to control the format and content of the data transmitted from 
the first system 1 to the second system 7. 

The second system 7 comprises one or more terminals capable of 
receiving and displaying streamed video data, such as desktop computers, laptop 
computers, mobile terminals and the like; in addition the second system 7 can 
comprise a data storage system DB for storing the streamed video data. The 
communications' network 5 can include a plurality of interconnected networks 
such as mobile networks (for example Public Land Mobile Network) and/or 
fixed line networks (for example one or more Integrated Services Digital 
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Networks and the Public Switched Telephone Network), in which case the 
environment 10 can also include various gateways (not shown) interconnecting 
these networks. The medical imaging device 3 is preferably a scanning device, 
such as an ultrasound machine, a magnetic resonance imaging (MRI) device or a 
5 computed tomography imaging (CTI) device and the first system 1 and second 
system 7 are connected via a public network or private network such as a Virtual 
Private Network (VPN), which may, or may not involve connection over the 
Internet. Alternatively the medical imaging device 3 could comprise camera- 
based equipment such as is used to perform endoscopy procedures. 
10 One particular application of embodiments of the invention is remote 

diagnosis and treatment of potential donor hearts for transplant. Referring to 
Figure 1, in such an application, video images of the heart of a potential donor 
are generated by an ultrasound machine 3 under control of an operator, and 
passed to the first system 1. The first system 1 processes the images into a 
15 format suitable for transmission to the second system 7, as is described in detail 
below, and then transmits the same to the second system, for example to a 
desktop computer Tl of the second system, thereby enabling an expert located at 
one of the clients of the second system to view a remote video image based on 
the ultrasound machine video output. The expert is able to send various control 
20 data to the first system, as is described in detail below, effectively guiding the 
operator through sufficient Trans Oesophageal Echocardiography (TOE) 
examination to perform clinical diagnosis so as to identify a heart's suitability 
for transplant and institute therapies to optimise the heart function prior to 
retrieval. Thus a system configured in accordance with embodiments of the 
25 invention can be used by an expert, who is located physically and logically 
remote from the medical imaging device 3, to control progress of a clinical 
examination. 

Referring now to Figure 2, features and functionality of the first system 1 

will now hr ripsrrihrri in more detail. Tho firct oyotom 1 can imnpuai une ui 

30 more computers, and in the representation shown in Figure 2 is embodied as a 
terminal comprising conventional computer components such as memory, CPU, 
Operating Systems (OS) programs, an Input/Output port and a fixed storage 
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disc, together with bespoke components arranged to manipulate, and ultimately 
transmit, data that originated from the ultrasound machine 3. More specifically, 
these components comprise an Analogue-to-Digital Converter 201, pre- 
encoding processing means 203, encoding means 205 and stream manager 207; 
at least some of these components are preferably embodied as a computer 
program, or a suite of computer programs, loaded onto the terminal, whilst 
others are embodied in hardware. 

Considering firstly the analogue-to-digital converter 201 (ADC), data 
received from the ultrasound imaging device are in standard video format, most 
commonly PAL (specifically PAL-I) or NTSC, over Composite or S-Video 
interfaces, and need to be digitised prior to any processing or encoding. The 
ADC 201 can be embodied as a capture card arranged to interface with the 
terminal 1 and capable of either digitizing a single video signal; or capable of 
digitizing and encoding a single video signal; or capable of digitizing and 
encoding multiple input video signals. Alternatively, the ADC 201 could be 
embodied in a capture unit external to the terminal 1, arranged to digitize a 
single video signal and transfer the digitized data, in real time, to the terminal. 
(An example of this would be the Canopus ADVC100, which digitises the S- 
Video or Composite video input, encodes the video into DV stream format (a 
very low loss, high bit rate, compression method) and streams this to the 
connected Media Server via a firewire (IEEE 1394) interface.) 

Having digitized the input video image data, the ADC 201 passes the 
data onto the pre-encoding processing means 203, an arrangement of which is 
shown in more detail in Figure 3. The steps carried out by pre-encoding 
processing means 203 are, broadly speaking, dependent on the type of 
information that is being, or likely to be, requested from the second system 7. 
For example, an expert associated with the second system 7 may wish to 
simultaneously view different regions of the heart, some in greater detail than 
others, to enable remote diagnosis and analysis of the condition of the heart. In 
such an example, a plurality of images could be requested by the second system 
7 - perhaps a first stream corresponding to the entire image captured by the 
medical imaging device 3 and a second stream corresponding to a specific 



region within the captured image - each at a different resolution, and/or frame 
rate. 

Accordingly, in one arrangement, the pre-encoding processing means 
203 replicates the image digitized by the ADC 201 so as to form a plurality of 
digitized images, and, for each of the plurality, selectively modifies the extents 
of the image region so that each corresponds to a different region of the image 
captured by the medical imaging device. The actual extents of these regions can 
either be pre-specified, specified by users of the second system 7 (i.e. by the 
doctor reviewing the image of the heart in real-time), or a combination of both, 
and are embodied in the form of image sizing parameters, as will be described in 
more detail below; the extents could also be specified by an operator associated 
with the first system 1 (i.e. local to the first system 1 and medical imaging 
device 3). Thus pre-encoding processing means 203 is responsive to data 
received from the second system 7 and data entered via a local input device 
associated with first system 1. 

In addition to performing the steps of replication, the pre-encoding 
processing means 203 can also filter the digitized images, so as to effect 
brightness adjustment, noise reduction, contrast/saturation management, and 
other types of image adjustment that will maximise the quality of images 
received and processed by the second system 7. 

Referring to Figure 4, an example of the steps carried out by the ADC 
201 and pre-encoding processing means 203 will now be described for the case 
where the pre-encoding processing means 203 creates three image portions 301, 
303, 305. At step 401, an image captured by the medical imaging facility 3 is 
received by the ADC 201, which digitizes the same to create a digital video 
image having frame size 720x576 and a frame rate of 25 frame-per-second (fps). 
The digitized image is then input to the pre-encoding processing means 203, 
which digitally resizes (step 403) the image to form a frame size of 360x288. 
Thiff ranged imago io then uocd to crcato a plurality of image puitioiu, (in tfrfe 
example 3) on the basis of specified image processing parameters. Referring to 
Figure 5, the image processing parameters 500 comprise sizing parameters 501, 
such as extents, centre and resolution of image portion, which are to be used 
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directly by the pre-encoding processing means 203, and image reproduction 
parameters 503, such as frame rate and data rate which are to be used by the 
encoding means 205. At least some of the image processing parameters 500 can 
be received from second system 2, in real-time, and stored in a buffer for 
retrieval by the pre-encoding processing means 203, as will be described in 
more detail below. 

Accordingly, at step 405, the pre-encoding processing means 203 
retrieves image processing parameters corresponding to each image portion 301, 
303, 305, replicates the cropped image three times, at step 407, and modifies 
each image (step 41 1) in accordance with the image processing parameters 500 
retrieved at step 407. Assuming the sizing parameters received at step 407 to be 
360x288; 260x120 at 100,120; and 100x80 at 50,50, the whole of the first 
replicated image is selected to form first image portion 301; a 160x120 section 
of the second replicated image located at 100,120 is selected to form second 
image portion 303; and a 100x80 section of the third replicated image located at 
50,50 is selected to form third image portion 305. Each image portion is then 
cropped (step 409) the image to a frame size of 352x288. Figure 4 does not 
include details of the possible filtering operations that can be performed (as 
described above, these can include brightness adjustment, noise reduction, 
contrast/saturation management), but the image portions could be so modified 
after replication (step 407) and prior to cropping (step 409). In addition, having 
filtered the image portions 301, 303, 305, they could be resized again; the 
skilled person will appreciate that the actual selection and implementation of 
these steps is a matter of design and likely to vary dependent on the type of 
image data. 

Each image portion 301, 303, 305 is then copied by a respective 
loopback driver 311, 313, 315 and input to encoding means 205 (a loopback 
driver is used in conjunction with the pre-encoding processing means 203 to 
ensure that the encoding means 205 receives an input from an video device 
driver; the skilled person will realize that this is a feature of the implementation 
of the encoding means 205 and not an essential feature of the invention). 
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In the arrangement shown in Figure 3, the encoding means 205 
comprises as many encoders (preferably embodied as one, or a suite of, 
computer programs) as there are image portions; however, it will be appreciated 
that there could be one encoder, which is arranged to receive each image portion 
5 and encode them in turn. In the present example, since there are three image 
portions, there are three encoders. Referring again to Figure 4, the pre-encoding 
processing means 203 passes image reproduction parameters 503 retrieved at 
step 407 (frame rate, data rate), together with any encoding-specific parameters, 
to a respective encoder for use in encoding a respective image portion at step 
10 413. The image portions can be encoded in accordance with any codec, such as 
Real Networks, Windows Media, Quicktime, each of which may comply with 
one or more of the codec technical standards (e.g. MPEG1, MPEG2, MPEG3, 
MPEG4, H.264 incorporated into MPEG4 as MPEG4 Part 10). In a preferred 
arrangement the image portions 301, 303, 305 are encoded in accordance with 
15 the MPEG4 standard, using MP4LIVE, which is a Linux™ audio/video capture 
utility that can capture and encode audio and video in real-time (for further 
details the reader is referred to the documentation associated with MP4LIVE, 
which, at October 2003, is managed by Dave Mackie and Bill May). 

Each encoder effectively carries out the same steps as the other encoders 
20 when encoding a respective image portion, but on the basis of a different set of 
image reproduction parameters. Continuing with the example above, and 
, assuming image reproduction parameters 503 corresponding to the first, second 
and third portions 301, 303, 305 are respectively 1 fps and lOkb/s; 2 fps and 
20kb/s; and 3 fps and 50kb/s, the encoding means 205 will generate (step 419) 
25 three MPEG video streams, having the following characteristics: 

A. A first stream of the whole captured image, of size 352x288, I fps 

B. A second stream, consisting of a medium size window, resolution 
260x120 of size 100,120, relative to a full screen resized to 352x288, at 2fps; 
aad 

30 C. A third stream, consisting of a small window, resolution 100x80 of 

size 50,50 relative to full screen resized at 352x288 at 3fps. 
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Thus the ADC 201, the pre-encoding processing means 203 and the 
encoding means 203 cooperate to receive, as input, an initial video image, 
digitize the same at 720x576 resolution and 25 fps, and generate, as output, 
encoded streams of images with different parameters. By choosing to encode 
data at different frame rates (in this example 1 fps for large image, 2 fps for 
medium and 3 fps for small image), prior to transmission to a remote location, 
the bandwidth required for these images is far lower than would be the case if 
the images were not processed prior to transmission. 

Having created multiple streams of images, each having different 
parameters associated therewith (and possibly having been filtered as described 
above), the streams are then transmitted to the second system 7 (step 415). 
Referring to Figures 2 and 6, transmission of data streams between the first and 
second systems 1, 7 is controlled by stream manger 207, which is embodied as 
one or a suite of computer programs arranged to receive, from the encoding 
means 203, a plurality of unicast Realtime Transport Protocol (RTP) streams 
and to serve the streams to terminals in the second system 7. In this particular 
arrangement, therefore, the stream manager 207 essentially acts as a 
communications broker between the first and second systems 1, 7. RTP, which 
is both an IETF Proposed Standard (RFC 1889) and an International 
Telecommunications Union (ITU) Standard (H.225.0), is a packet format for 
multimedia data streams generally used in conjunction with Real-Time 
Streaming Protocol (RTSP), which is a control protocol responsible for initiating 
and directing delivery of streaming multimedia from media servers such as the 
first and second systems 1, 7, In addition to standard RTP control data, RTSP is 
used to carry non-protocol information, such as sizing parameters 503 
(positional and size information) associated with the various image portions 301, 
303, 305; this positional information is used by the second system 7 when 
creating a single image from the multiple received video streams, as will be 
described in detail below. Alternatively, the Session Initialization Protocol 
(SIP), one of the ITU protocols, or a combination of RTSP and SIP could be 
used as the control protocol. 
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It should be noted that streams modified in accordance with 
embodiments of the invention require significantly less bandwidth, whilst still 
providing users of the second system 7 with sufficiently detailed images at 
locations of interest, than unmodified streams. For certain image processing 
5 parameters 500, reductions in bandwidth usage of 66% or more have been 
achieved. 

The stream manager 207 is configured such that data can be captured by 
the medical imaging facility 3, processed and passed to the stream manager 207 
on a continuous basis and thus independently of any requests, or lack thereof, 

10 received from the second system 7. In effect, in the absence of requests for a 
stream from the second system 7, all external RTP transmission of streams is 
"muted" and no data are sent from the first system 1; this means that external 
bandwidth is only used when necessary. The skilled person will realize, 
however, that there are alternative ways of controlling the transmission of 

15 encoded data streams, for example, by selectively encoding (and immediately 
transmitting) the digitized data in response to a request from the second system 
7. 

Remote selection of the image processing parameters 500 at the second 
system 7 will now be described. Any set of image processing parameters 500 
20 can conveniently be described as a mode and a plurality of mode options, so 
that, when a primary user, that is to say a clinician directing the diagnosis, at 
terminal Tl wants to view a different region and/or at a different frame rate 
and/or resolution etc. he selects different mode options (to be described in more 
detail below). These mode options, in the form of image processing parameters, 
25 are propagated through the network 5 to the first system 1 and stored locally, for 
retrieval by the pre-encoding processing means 203 at step 407 and transmission 
on to secondary clients T2, T3 (passively reviewing the medical images) 
requesting image data in the second system 7 at step 417. Alternatively the 

pnrnmrterg 500 can bo stored ocnfraHy-foot-shuwii) aad acceded by the 

30 secondary clients of the second system 7 when submitting their request for 
imaging data to the first system 1. As a further alternative the parameters 500 
couid be transmitted to, and stored by, the secondary clients T2, T3 directly. 
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Preferably the RTSP control channel, SIP or one of the ITU protocols is used to 
send data indicative of a change in mode option. 

Receipt and display of the image streams at a client terminal Tl of the 
second system 7 will now be described with reference to Figures 7, 8 and 9. In 
addition to conventional operating system software, I/O port, video card, hard 
drive, CPU and memory, the terminal Tl comprises a plurality of software 
components that cooperate to receive, decode and process data streams received 
from the first system 1 for display by terminal Tl. As described above, the 
streams comprise RTP and RTSP (and/or SIP, and/or other ITU protocols) data, 
which are received by the terminal Tl, in accordance with standard methods, 
and passed to decoding means 703, which decodes (step 801) the streams in 
accordance with whatever codec was used in step 413 to encode the image data 
(here MP4LIVE) and passes the decoded streams to a multiplexer 705. The 
multiplexer 705 is responsible for copying the decoded image frames from the 
input streams to form a single final image frame for display, using the data 
specifying the relative positions of the video images sent using RTSP control 
channel. In a typical scenario, three streams are active: (i) a background stream, 
showing semi-static text around the screen; (ii) a large "window" with a moving 
image in the lower half of the screen, showing a slow moving image; and (iii) a 
small "window" at the top right of the screen containing a fast moving image. 

For clarity, and referring to Figure 8, the steps carried out by the 
multiplexer 705 will be described with reference to streams A, B, C. The 
multiplexer 705 reads in the relative positions of the streams A, B, C (e.g. from 
the RTSP data - at step 803), and selectively associates each stream with a 
location on a display area G (Figure 9). One suitable method involves use of a 
mask having a plurality of regions Rl, R2, R3, each of which is associated with 
an input stream A, B, C, so that, at step 805 the multiplexer 705 copies the 
pixels from the input video streams into an output buffer based on the associated 
mask identity - for example, the region Rl having a lowest identifier is 
associated with the first stream (which in this example is stream A). Preferably, 
and as shown in Figure 9, each region Rl, R2, R3 has a colour associated 
therewith, so that the region having a lowest RGB value (Rl) is associated with 
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stream A, the region having the next lowest RGB value (R2) is associated with 
stream B etc. This is advantageous since, when the contents of the output buffer 
are displayed, the location of image portion of an input stream relative to its 
intended location within the display area G can be easily identified (since, if an 
5 image were offset relative to its associated mask region Rl (thus running the 
risk of effectively deleting some of the image), this would be immediately 
apparent). Accordingly the multiplexer 705 is arranged to apply an offset to 
each of the image portions on the basis of the position information received on 
the RTSP control channel - for example changing the position of the image 
10 portion corresponding to stream B to 95,120 - for the image portions to fit 
within their respective mask region Rl, R2, R3. Any additional colours on the 
mask are assigned black pixels so as to minimize edge effects around the 
images. 

Having copied the pixels from the input video streams into an output 
15 buffer based on the associated mask identity, the multiplexer 705 passes the 
output buffer to a renderer 707, which, in accordance with known methods, 
creates an image on the display 71 1 based on data in the output buffer (step 
807). Conveniently the image rendered at step 807 is presented within a 
Graphical User Interface (GUI) comprising a plurality of selectable mode 
20 options, each corresponding to a different set of imaging parameters 500 (to be 
described in more detail below); preferably the options are prespecified, but they 
could be.user-configurable. 

Before being sent to the renderer 707, the multiplexer 705 preferably 
links the output buffer (or more specifically the co-ordinates of pixels in the 
25 output buffer) to an overlay layer, which, when displayed by the GUI is 
responsive to inputs from the user and is arranged to capture user-specified 
regions of interest and markers placed within the rendered images. The renderer 
707 is arranged to receive data indicative of modified regions of interest (step 
S OB), transform th e m into oo - ordinate^ mid/ui-ttbulutiuii valuta and transmit to 
30 the first system 1 as modified image processing parameters (step 811). 

Since, as described above, the imaging processing parameters 500 can be 
modified at any time (e.g. by a change of mode options - step 809), new data 
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streams, corresponding to different image portions and/or different frame rates, 
data rates and resolutions, can be sent from the first system 1 to client terminal 
Tl at any time. The multiplexer 705 thus has to co-ordinate receipt and 
processing of incoming streams to ensure that a coherent image is presented on 
5 the display 71 1 (i.e. that the frame rendered in region Rl corresponds to that in 
region R2 and R3). In one arrangement, the multiplexer 705 has access to a 
local store (e.g. memory 713) and a configurable timer, preferably arranged to 
run at 25 times a second; the skilled person will appreciate that frame rate is 
dependent upon the type of video format, so that, for example an 
10 implementation of the system using an NTSC signal for its video source may 
have a different frame rate to that of one using a PAL signal source. The 
multiplexer 705 maintains copies of the most recently received video frame for 
each input stream, and, when the timer is triggered, combines the stored frames 
into the output buffer as described above (steps 803, 805). Alternatively steps 
15 803 and 805 could be performed in response to receipt of input video frames, 
but this approach could lead to a less cohesive rendering of the display, since 
frames from different streams can be received at different times. 

The various modes of operation for a particular medical imaging device, 
namely an EchoCardioGraph unit, will now be described. Conventional 
ultrasound scanners create two-dimensional B-mode (also referred to as Echo 
mode or Blood mode) images of tissue in which the brightness of a pixel is 
based on the intensity of the echo return and/or, in a color Doppter mode, the 
movement of fluid (e.g., blood) or tissue. Measurement of blood flow in the 
heart and vessels using the Doppler effect is well known - the phase shift of 
backscattered ultrasound waves may be used to measure the velocity of the 
backscattered waves from tissue or blood, and this Doppler shift can be 
displayed using different colors to represent speed and direction of flow. 
Alternatively, in power Doppler imaging, the power contained in the returned 
Doppler signal is displayed. 

When operating in accordance with embodiments of the invention in the 
B-mode, a Region of Interest, frame rate, resolution and data rate can be 
specified, via the plurality of selectable mode options described above, for 
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different parts of a B-mode image. Referring back to Figures 3 and 9, in the 
mode, a first image portion 301 (i.e. contained within stream A and rendered in 
region Rl) can be a background image containing data identifying the settings 
of the echocardiograph machine 3, which can be expected to remain static over 
time and thus transmitted at a low frame rate and data rate; a second image 
portion 303 (i.e. contained within stream B and rendered in region R2) can be an 
image of the entire B-mode image area, which varies over time and needs to be 
captured and transmitted at a faster frame and data rate respectively than that 
corresponding to image portion 301; and a third image portion 305 (i.e. 
contained within stream C and rendered in region R3) can be a Region of 
Interest within second image portion 303, jspecified by a user connected to the 
second system 7 and at a faster frame and data rate still, since this is the region 
for which the user requires more detail. As an alternative to the arrangement 
shown in Figure 9, region R3 can be superimposed upon region R2. 

The ECG unit can operate in a number of alternative modes, such as 
Continuous Wave (CW) mode, in which a graphical representation of cardiac 
activity is displayed, from a specific location in the heart, together with an 
image of the region of the heart at that location. Referring back to Figures 3 and 
9, a first image portion 301 (i.e. contained within stream A and rendered in 
region Rl) can be a background image containing data identifying settings of 
the echocardiograph machine 3, which can be expected to remain static over 
time and thus transmitted at a low frame rate and data rate; a second image 
portion 303 (i.e. contained within stream B and rendered in region R2) can 
correspond to a graph describing the magnitude of received echoes, while a third 
image portion 305 (i.e. contained within stream C and rendered in region R3), 
contains the image of the region of the heart at which ultrasound echoes are 
being measured. In this mode of operation the second image portion 303 in 
region R2 is the focal point and the frame rate and data rate corresponding 
therrtnifi typiralty selected to be fnoter than those corresponding tu the fu^L and 
third image portions 301 and 305; these mode options can be selected via a 
plurality of selectable mode options on the GUI. 
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Additional deta ils 

In addition to streaming video data between the first and second systems 
1, 7, terminals Tl, T2 of the second system 7 can also include a Voice Over IP 
(VOIP) and so be arranged to receive and transmit audio data. Audio input can 
be received using a standard microphone input associated with terminal Tl and 
encoded using a speech codec such as Speex 1 lk wideband codec. The encoded 
speech data could then be transmitted to a central conference point (not shown) 
using RTP, or alternative protocol, which comprises software arranged to 
decode and combine incoming audio streams, then re-encode the same and 
transmit to all clients receiving data from the first system 1. The conference 
point could be provided as part of the first system 1 or implemented in a server 
outside of both the first and second systems. Irrespective of where the 
conference point is actually running, the re-encoded streams could be passed 
within (or transmitted to) the first system 1 and multiplexed with the video 
15 streams in a transport stream, prior to transmission to the second system 7. 
Alternatively, the re-encoded streams could be transmitted directly to all clients 
Tl, T2 from the conference point. 

When received at clients Tl, T2 of the second system 7, the (combined) 
audio stream is received within the client transport stack and passed to the 
decoding means 703, which additionally includes means for decoding audio 
data, and optionally echo cancellation (echo cancellation may be required within 
the system dependent upon the overall delays, and whether the user (expert, 
operator) is using a headset or standard speakers for the audio of the system). 
Once decoded by the decoding means 703, the audio data can be output via a 
25 standard PC audio output device. 

In addition to the video and audio, the environment 10 can be arranged 
to capture and display data from other medical equipment. This is likely to 
include (but not be limited to) Vital Signs Monitors, Pulse Oximeters and 
ElectroCardioGraph (ECG) Waveforms, so that the first system 1 preferably 
includes at least some of the following interfaces: RS232, GPIB/HPIB or 
equivalent, Raw Waveforms (analogue), Visual data, Digitisation of data. Any 
data obtained via RS232, GPIB/HPIB etc will already be in digital form, and 
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consequently only need processing to extract a required subset from any data 
stream, packetisation and/or header information supplied by the medical 
equipment. Raw waveforms required for transmission and output (visual* audio 
or otherwise) must be digitized, e.g. through standard audio input if the 
5 bandwidth and sample rates allow, or alternatively through use of DSPs and 
sampling, and encoded using codecs to minimise bandwidth requirements. For 
data that are only be available visually, i.e. as part of a video output (e.g. an 
ECH waveform on an ultrasound display), the first system 1 can include means 
arranged to extract a waveform from video data, for encoding and transmission 
1 0 for display or output. 

Whilst in the above described embodiments the pre-encoding processing 
means 203 is described as being separate from the encoding means 205, they 
could be combined. 

Whilst in the above described embodiments all of the images encoded by 
1 5 the encoding means 205 are sent to terminals Tl, T2, T3 of the second system 7, 
alternatively the first system 1 could be arranged to stream only selected image 
portions 301, 303, 305 to the terminals Tl, T2, T3. The stream manager 207 
could, for example, be selectively responsive to requests from the terminals, in 
accordance with specified streaming requirements. Such streaming 
20 requirements could, for example, be transmitted from individual terminals Tl, 
T2, T3 of the second system 7, so that individual terminals can select which 
streams they receive; alternatively the primary terminal Tl could relay streams 
transmitted thereto onto requesting secondary terminals T2, T3 in accordance 
with specified streaming requirements. 
25 Whilst in the above described embodiments the stream manager 207 is 

arranged to transmit the image data as unicast streams using RTP and 
RTSP/SIP, the stream manager 207 could alternatively be arranged to transmit 
the image data as multicast packets, in which case nodes Tl, T2, T3 of the 
seronrf Byfitpm 7 could be arranged to oemri "join" rcqucjtj wliui iiqui&ling 
30 image data. 

Whilst in the above embodiments the second system 7 is described as 
including a data storage system DB for storing the streamed video data, the first 
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system 1 may additionally or alternatively store the video data, at high quality, 
for deferred playback. 

The above embodiments are to be understood as illustrative examples of 
the invention. It is to be understood that any feature described in relation to any 
5 one embodiment may be used alone, or in combination with other features 
described, and may also be used in combination with one or more features of 
any other of the embodiments, or any combination of any other of the 
embodiments. Furthermore, equivalents and modifications not described above 
may also be employed without departing from the scope of the invention, which 
1 0 is defined in the accompanying claims. 
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Claims 

1 • A method of transmitting medical image data from a first system 
5 attached to a communications network to a second system receiving medical 
image data, the second system being remotely connected to the first system via 
the communications network, the method comprising the steps of: 

receiving first image data from a medical imaging device, the first image 
data having a video format; 

10 receiving data identifying one or more image processing parameters, 

from the second system; 

in response to receipt of said one or more image processing parameters, 
selectively processing the first image data in accordance with said one or more 
image processing parameters so as to create second image data at the first 

15 system; and 

transmitting the second image data to the second system. 

2. A method according to claim 1, in which the first image data 
comprise analogue video format data, the method including converting the 
20 analogue video format data into initial digitised data, and selectively processing 
the initial digitised data so as to create said second image data. 



3. A method according to claim 1 or claim 2, in which the image 
processing parameters include one or some of frame rate, resolution, and/or data 

25 rate. 

4. A method according to any one of the preceding claims, in which 
the image processing parameters include data specifying extents of an area of 

interest within ealA firnf i^Q- data 

30 

5. A method according to any one of the preceding claims, in which 
the second image data comprise a plurality of digitised images. 
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6. A method according to claim 5, in which each digitised image is 
processed in accordance with a different set of image processing parameters. 

7. A method according to claim 6 when dependent on any one of 
claim 3 to claim 5, in which the frame rate associated with one of the sets of 
image processing parameters is faster than that associated with at least one other 
of the plurality. 

8. A method according to claim 6 or claim 7 when dependent on 
any one of claim 3 to claim 5, in which the resolution associated with one of the 
sets of image processing parameters is finer than that associated with at least one 
other of the plurality. 

9. A method according to any one of claim 6 to claim 8 when 
dependent on claim 4, in which a first area of interest associated with one of the 
sets of image processing parameters is different to a second area of interest 
associated with at least one other of the plurality. 

10. A method according to claim 9, in which the first and second 
areas overlap. 

11. A method according to claim 9 or claim 10, in which the first 
area is larger than said second area, 

12. A method according to any one of the preceding claims, 
including receiving data identifying one or more image processing parameters 
from the first system. 



13. A data processing system for transmitting medical image data 
through a communications network, the data processing system comprising: 
a first system attached to the communications network; and 
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a second system remotely connected to the first system via the 
communications network; 

wherein the first system comprises: 

data receiving means arranged to receive first image data from a 
medical imaging device, and image processing data identifying one or 
more image processing parameters; 

data processing means arranged, in response to receipt of said 
one or more image processing parameters, to selectively process the 
received image data in accordance with one or more image processing 
parameters so as to create second image data; and 

data transmitting means arranged to transmit the second image 
data to the communications network, 

and wherein the second system is arranged to transmit said image 
processing parameters to the first system and receive said second image data 
transmitted by the first system. 

14. A data processing system according to claim 13, wherein the data 
processing means is arranged to create a plurality of images corresponding to 
said first image data. 

15. A data processing system according to claim 13 or claim 14, 
Wherein the second system is arranged to transmit data identifying a plurality of 
sets of one or more image processing parameters specifying characteristics of a 
plurality of images. 

16. A data processing system according to claim 15 when dependent 
on claim 14, wherein, for each created image, the data processing means is 
arranged to process said image in accordance with a different set of image 
processing paramrterg received from the aeo o n d o yotcm, said sciund image data 
thereby comprising a plurality of differently processed images. 



24 



17. A data processing system according to any one of claim 13 to 
claim 16, wherein said second system comprises a plurality of nodes. 

18. A data processing system according to claim 17, wherein a first 
node of the plurality is arranged to transmit the image processing parameters 
and a second node of the plurality is arranged to receive at least part of the 
second image data. 



19. A data processing system according to claim 18, wherein said 
first node of the second system is arranged to transmit data corresponding to the 
image processing parameters to the second node, for use in processing said 
second image data when received by said second node. 

20. A data processing system according to any one of claim 17 to 
claim 19, wherein the plurality of nodes in the second system includes a 
plurality of user terminals, each attached to a different part of the 
communications network. 

21. A data processing system according to claim 20, wherein each of 
the plurality of user terminals is arranged to receive at least part of the second 
image data transmitted from the first system. 

22. A data processing system according to claim 2 1 , wherein the first 
system and second system are inter-connected via a Virtual Private Network. 

23. A data processing system according to any one of claim 17 to 
claim 22, wherein the, or each, node in the second system is associated with a 
physician. 



24. A data processing system for use in displaying medical images, 
the data processing system comprising: 
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data receiving means arranged to receive a plurality of data streams and 
image processing parameters corresponding thereto, each data stream 
corresponding to a medical image; 

display means arranged to display a display mode selected from a 
5 plurality of display modes, the display modes comprising a display mode 
having at least two regions each corresponding to one of said received data 
streams, 

wherein the system is arranged to select a display mode in dependence 
on the image processing parameters. 

10 

25. A data processing system according to claim 24, wherein, for 
each data stream, the image processing parameters include location data 
specifying the extents and position of the medical image corresponding thereto, 
and the display means is arranged to select a display mode in dependence on the 

15 location data. 

26. A data processing system according to claim 24 or claim 25, 
wherein, in at least one display mode, at least part of the regions overlap with 
one another. 

20 

27. A data processing system according to any one of claim 24 to 
claim 26, including at least one multimodal input device, and wherein the 
display means is arranged to receive data indicative of a portion of interest via 
the multimodal input device. 

25 

28. A data processing system according to claim 27, wherein the 
system is arranged to transmit said data indicative of a portion of interest to a 
terminal located remote from the data processing system, for use thereby in 

Mtnt'mp said mndinal images 

30 

29. A medical imaging system for transmitting medical image data 
through a communications network, the medical imaging system comprising: 
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a medical imaging device arranged to generate first image data in respect 
of an object being imaged; 

a first system attached to the communications network; and 
a second system remotely connected to the first system via the 
communications network; 

wherein the first system comprises: 

data receiving means arranged to receive said first image data 
from the medical imaging device, and image processing data identifying 
one or more image processing parameters; 

data processing means arranged, in response to receipt of said 
one or more image processing parameters, to selectively process the 
received image data in accordance with one or more image processing 
parameters so as to create second image data; and 

data transmitting means arranged to transmit the second image 
data to the communications network, 

and wherein the second system is arranged to transmit said image 
processing parameters to the first system and receive said second image data 
transmitted by the first system. 

30. A data processing system for use in generating medical images 
for transmission to one or more terminals logically located remote therefrom, 
the data processing system comprising: 

receiving means arranged to receive data identifying one or more image 
processing parameters; 

image processing means arranged to modify first image data having a 
video format in accordance with said one or more image processing parameters 
so as to create second image data; and 

transmitting means arranged to transmit said second image data to said 
one or more terminals. 
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31, A data processing system according to claim 30, wherein the 
receiving means is arranged to receive said image processing parameters from 
said one or more terminals. 

5 32, A method of transmitting image data from a first system attached 

to a communications network to a second system receiving image data, the 
second system being remotely connected to the first system via the 
communications network, the method comprising the steps of: 
receiving first image data having a video format; 
10 receiving data identifying one or more image processing parameters, 

from the second system; 

in response to receipt of said one or more image processing parameters, 
selectively processing the first image data in accordance with said one or more 
image processing parameters so as to create second image data at the first 
15 system; and 

selectively transmitting the second image data to the second system. 
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